💡 Codex Review

Stamp document-mode chunks with their own chunker version

When CHUNK_STRATEGY=document, emitChunk still writes this constant into metadata.chunker_version, so shadow re-index rows produced with cross-page boundaries are labelled 1.0.0-page. Any stale-index/cutover/eval logic that relies on this metadata cannot tell page-bounded chunks from document-mode chunks, even though their chunk boundaries and source spans differ; make the version strategy-specific or include the strategy in metadata.

Restrict document-mode image IDs to contributing pages

In document mode, a multi-page section passes all sectionImages into every emitted chunk, while emitChunk can attach an image based on section-path/header matches rather than the chunk text. For a long section with a table on a later page, an earlier text-only chunk can be stored with that unrelated table's image_ids, causing retrieval results/citations to show visual evidence from pages that did not contribute to the chunk; pass only images from contributing pages or otherwise page-filter them here.

Build per-page source span excerpts for cross-page chunks

For document-mode chunks spanning multiple pages, every sourceSpanForText call receives the full cross-page excerpt. The first page may match, but later contributing pages search for text that starts on an earlier page and usually end up with null offsets, so citations cannot anchor to those pages; derive a page-specific excerpt (or split the chunk text by contributing page) before creating each span.

💡 Codex Review

Derive page-local spans for cross-page chunks

When CHUNK_STRATEGY=document emits a chunk spanning multiple pages, this calls sourceSpanForText for each contributing page with the same full-chunk excerpt. Because sourceSpanForText searches the first part of that excerpt in the page text, only the first page of the chunk can match; later pages get null offsets and an excerpt that starts with text from a different page. This breaks source anchors/highlighting for evidence that came from a continuation page, so the span should be built from a page-local snippet before storing each entry.

💡 Codex Review

Build page-specific spans for cross-page chunks

When CHUNK_STRATEGY=document creates a chunk spanning multiple pages, this maps every contributing page through sourceSpanForText with the same full-chunk excerpt. Because sourceSpanForText only searches from the beginning of that excerpt, later pages usually get null offsets and an excerpt copied from page 1, so source metadata/citations for evidence that came from page 2+ point at the wrong text. Pass a page-specific excerpt (or split the chunk text per contributing page) when building each span.

Stamp document chunks with the active strategy version

The new document-mode chunker still stamps every row with CHUNKER_VERSION = "1.0.0-page". In the shadow re-index path that runs with CHUNK_STRATEGY=document, staged cross-page chunks become indistinguishable from page-bounded chunks in metadata, defeating the stated purpose of using the version to identify which chunker produced a row and to invalidate stale chunks after boundary changes. Include the active strategy/params in the stamped version before using document mode for eval-gated re-indexes.

Limit document-mode images to contributing pages

In document mode this passes every image from the whole section into emitChunk, even when attributeChunkToPages has narrowed the chunk to only one or two contributing pages. For long sections with multiple tables/images, the heading/path matching in emitChunk can attach image_ids from pages outside page_start/page_end, so source-image/table requests may treat an unrelated page's visual as evidence for the chunk; pass only images from the contributing pages.

Scope document-mode dedupe beyond raw content

With CHUNK_STRATEGY=document, this document-wide fingerprint drops every later chunk whose normalized content matches an earlier one, regardless of page or section. Identical clinical rows or repeated instructions can appear under different headings/pages and need distinct source attribution; page mode preserved those duplicates, but document mode now silently removes them from the staged index. Include section/page context in the dedupe key or restrict this to proven boilerplate.

💡 Codex Review

Stamp document-mode chunks with their own version

When CHUNK_STRATEGY=document is enabled for the shadow re-index, buildDocumentModeChunks still reaches this shared emitter and stamps chunker_version with the constant value 1.0.0-page, with no separate strategy marker in the metadata. That makes document-mode rows indistinguishable from the default page-mode rows, so any stale-row/re-index health check using the advertised chunker version cannot tell whether the candidate was actually produced by the cross-page chunker; include the active strategy in the version or metadata.

Replace literal NUL separators with escaped text

This template literal contains actual 0x00 bytes in the TypeScript source, which makes Git classify src/lib/chunking.ts as binary and hides textual diffs for the whole file in normal review/patch tooling. The hash can still use NUL separators without making the source binary by writing them as escaped \0 characters instead.

Preserve page ranges for cross-page chunk citations

When document-mode chunking merges content from pages 1–2 and the answer comes from the later page, this stores only the first contributing page as the chunk's page_number. The citation path uses result.page_number for labels and does not read metadata.page_start/page_end, so users can be sent to page 1 for evidence that came from page 2; propagate a page range into citations or avoid collapsing cross-page chunks to a single representative page.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 160ce6c755